Methods for generating three-dimensional image data of human bones

ABSTRACT

The present invention relates to a method for generating three-dimensional image from two-dimensional images, and more specifically, a method for generating three-dimensional image of human bones from two 2D planar images thereof. The method comprises the steps of: providing a first X-ray planar image and a second X-ray planar image; predicting one set of predicted posture parameters for each of the first X-ray planar image and the second X-ray planar image; and generating the data of a stereoscopic image according to the first X-ray planar image, the second X-ray planar image, and the predicted posture parameters. The present invention also relates to a method for training an artificial intelligence to perform three-dimensional image generation described above.

RELATED APPLICATION

This application claims the benefit of the Taiwan Patent Application No.110120781 filed on Jun. 8, 2021, titled “METHOD AND DEVICE FORGENERATING THREE-DIMENSIONAL IMAGE DATA OF HUMAN BODY SKELETAL JOINTS,”which is incorporated herein by reference at its entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a method and device for generatingstereoscopic image data, especially a method and device for generatingstereoscopic image data of human bones and joints from two-dimensionalimages.

Description of Related Art

Nowadays, 3D image reconstruction is an important tool for diagnosingbone-related diseases in field of medicine. Presently, the mosteffective and widely used 3D imaging technology is computed tomography,referred to as CT. Computed tomography is an accurate 3D imagingtechnique that produces high-resolution information about the internalstructure of the human body. However, the multiple X-ray exposures of CTscans result in high radiation doses to patients, and CT scanners arerelatively expensive and too bulky to move. Although some 3D imageconstruction methods are currently available and enable construction ofstereoscopic images from two planar images, none of them can providesatisfactory results for human bones and joints, especially when the twoinput planer images are not positioned orthogonally.

Therefore, it is desirable to develop a new method to generate astereoscopic image from two-dimensional images such as X-ray images.

SUMMARY OF THE INVENTION

To resolve the problems, the present invention provides a method forgenerating three-dimensional image of human bones, comprising the stepsof: providing a first X-ray planar image and a second X-ray planar imagecaptured from a first angle and a second angle of the human bone;predicting, by an image processing engine, one or more sets of predictedposture parameters for the first X-ray planar image and the second X-rayplanar image; and generating, by the image processing engine, the dataof a stereoscopic image according to the first X-ray planar image, thesecond X-ray planar image, and the predicted posture parameters.

In one embodiment, before predicting posture parameters the methodfurther comprising removing a first interference image from the firstX-ray planar image and removing a second interference image from thesecond X-ray planar image, wherein each of the first interference imageand the second interference image is the background interfering imageresulting from non-skeletal objects.

In one embodiment, the image processing engine utilizes a machinelearning algorithm, one or more known three-dimensional (3D) validatingimages, and a plurality of two-dimensional (2D) training imagesgenerated from the known 3D validating images to optimize the ability togenerate the data of a stereoscopic image. The plurality of 2D trainingimages may be digitally reconstructed from projecting the known 3Dvalidating images at different angles. In a specific embodiment, theknown 3D validating images are CT images.

In one embodiment, the machine learning algorithm is a convolutionneural network (CNN).

In one embodiment, the plurality of 2D training images comprises a setof training posture parameters. The training posture parameters maycomprise the rotation angles for each of the plurality of 2D trainingimages around x, y and z axis, and may also comprise an angle θrepresenting the bending angle of a joint.

In one embodiment, the predicted posture parameters comprise a set offirst predicted posture parameters for the first X-ray planar image anda set of second predicted posture parameters for the second X-ray planarimage. The predicted posture parameters may comprise the rotation anglesfor each of the first X-ray planar image and the second X-ray planarimage around x, y and z axis. Also, it may further comprise an angle θrepresenting the bending angle of a joint in the first X-ray planarimage and the second X-ray planar image.

Another aspect of the present invention is to provide a machine learningmethod for training a machine with image data of human bones, comprisingthe steps of: providing one or more three-dimensional (3D) validatingimage associated with human bones; providing a plurality oftwo-dimensional (2D) training images, each of which is a projected imagegenerated from one of the 3D validating images in a specific angledefined as an angle parameter associated with the 2D training image; andtraining the machine with the 3D validating images, the plurality of 2Dtraining images, and the angle parameters associated with the pluralityof 2D training images, wherein the machine after training is able togenerate a 3D target image from two 2D input images representingdifferent projections of the 3D target image.

Other objectives, advantages and novel features of the invention willbecome more apparent from the following detailed description when takenin conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is the block diagram of a preferred embodiment of a stereoscopicimage data generating apparatus developed in the present application.

FIG. 1B is a schematic diagram of the first angle defined in thisapplication.

FIG. 1C is a schematic diagram of the second angle defined in thisapplication.

FIG. 1D shows a skeletal planar X-ray image with a relatively cleanerskeletal image.

FIG. 2 is a step flow chart of the developed method for generatingstereoscopic image data of bones.

FIG. 3A shows the posture parameter of the joint bending angle θ in thepresent application.

FIG. 3B shows the posture parameters of the three-axis' rotationalangles (x, y, and z) of the knee joint in the present application.

FIG. 4 is the block diagram of another preferred embodiment of thestereoscopic image data generating apparatus developed in the presentapplication.

FIG. 5 shows the process of interference image removal in the example.

FIG. 6 shows the reconstructed 3D image quality improved by interferenceimage removal. FIG. 6A is the CT reference which generates the two inputimages, and

FIG. 6B-6E are the constructed 3D images using the two generated inputimages.

FIG. 6B is the construction result of two 2D X-ray images with bothinterference images retained; FIG. 6C is the construction result withthe interference image of anterior-posterior view removed; FIG. 6D isthe construction result with the interference image of lateral viewremoved; and FIG. 6E is the result where both interference images areremoved. The multi-scale structural similarity (MS-SSIM) values are alsoprovided below each figure.

FIG. 7 shows an example of generated training 2D surface with labeledposture parameters.

FIG. 8 shows the first embodiments to implement AI training forconstructing stereoscopic image.

FIG. 9 shows the second embodiments to implement AI training forconstructing stereoscopic image.

FIG. 10 shows the third embodiments to implement AI training forconstructing stereoscopic image.

FIG. 11 shows the reconstructed 3D image quality influenced bynon-orthogonal inputs and improved by consideration of postureparameters. FIG. 11A is the CT reference used to generate input images,and FIG. 11B-11D are the output 3D images. FIG. 11B is an output 3Dimage constructed from two orthogonal input images; FIG. 11C is anoutput 3D image constructed from two non-orthogonal input images withoutconsideration of the posture parameters; FIG. 11D is an output 3D imageconstructed from two non-orthogonal input images with consideration ofthe predicted posture parameters. The MS-SSIM values are also providedbelow each figure.

FIG. 12 shows MS-SSIM performance versus rotation angle for 3D imagesconstructed from two planar images with different angles. The anglevalue shown on X-axis is the angle deviates from orthogonal (90degrees).

FIG. 13 shows a comparison between generated 3D image and original CT 3Dimage, with MS-SSIM value 0.81641. The two input images generated fromthe CT 3D image are also provided below.

FIG. 14 shows the construction result (C and D) from the input images (Aand B). The two images on the right (C and D) are the generatedthree-dimensional image with intensity information. The whiteness(opacity) in the images represents the intensity values.

REFERENCE SIGNS

-   10: Image processor-   11: Image display-   131: First X-ray planar image-   132: Second X-ray planar image-   1310: First interference image-   1320: Second interference image-   133: Skeletal planar image-   141, 142: arrows-   19: Three-dimensional object forming device-   Step 21-24: Process steps-   30: Femur-   31: Tibia-   θ: Angle-   X, Y, Z: Coordinates

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The terminology used in the description presented below is intended tobe interpreted in its broadest reasonable manner, even though it is usedin conjunction with a detailed description of certain specificembodiments of the technology. Certain terms may even be emphasizedbelow; however, any terminology intended to be interpreted in anyrestricted manner will be specifically defined as such in this DetailedDescription section.

The embodiments introduced below can be implemented by programmablecircuitry programmed or configured by software and/or firmware, orentirely by special-purpose circuitry, or in a combination of suchforms. Such special-purpose circuitry (if any) can be in the form of,for example, one or more application-specific integrated circuits(ASICs), programmable logic devices (PLDs), field-programmable gatearrays (FPGAs), graphics processing units (GPUs), etc.

In the present application, an apparatus and method for generatingthree-dimensional image data is provided. A block diagram of a preferredembodiment of the apparatus is shown in FIG. 1A. The apparatus disclosedin this application can be applied to the reconstruction of stereoscopicimages and density of human skeleton and joints. It comprises an imageprocessor 10 and further comprises an image display 11 in oneembodiment. The image processor 10 can perform the method for generatingstereoscopic image data of bones as shown in the flowchart of FIG. 2 ,wherein step 21 is to receive a first X-ray planar image 131 and asecond X-ray planar image 132 captured from a first angle and a secondangle of the human bone, respectively. Each of the first angle and thesecond angle refers to a specific angle for photographing the humanbones and joints (for example, the arrows 141 and 142 shown in FIG. 1Band FIG. 1C respectively and the front of the right knee arerespectively formed two angles). The best angle between the first angleand the second angle is 90 degrees but other angles are also possible.

Step 22 is to remove a first interference image 1310 caused bynon-skeletal objects in the first X-ray planar image 131, and a secondinterference image 1320 caused by non-skeletal objects in the secondX-ray planar image 132. Most of the above-mentioned non-skeletal objectsare soft tissues (such as human muscles) or non-human tissues (such asclothing or surgical implants). The method in step 22 for removing theinterfering images caused by non-skeletal objects may be an existingimage processing method (e.g., an image background removal algorithm),which automatically removes the images generated by the non-skeletalobjects, so as to generate a cleaner skeletal image (the skeletal planarimage 133 as shown in FIG. 1D). The interference image removal method(e.g., the image background removal algorithm) may also be trained by anartificial intelligence technology to increase its accuracy.

Step 23 is to perform image processing on the first X-ray planar imageand the second X-ray planar image after removing the interferenceimages. The image processing may generate a stereoscopic image data fileaccording to the first X-ray plane image and the second X-ray planarimage after removing interference images, and may generate a pluralityof planar images with different angles from the stereoscopic image datafile (step 24). As for the image display 11, it may electrical connectedto the image processor 10 by signal to receive the plurality of planarimages of different angles generated by the stereoscopic image data fileand display them respectively, so that the viewer can see the imageswith stereoscopic sense.

In one embodiment, in order to reduce the burden of computing resources,only one of the first X-ray planar image and the second X-ray planarimage is selected to remove the corresponding interference image (i.e.first interference image 1310 or second interference image 1320), andthe other X-ray planar retains its interference image. Even so, afterthe image processing is performed, a stereoscopic image data file withquality better than the prior art can still be obtained.

The image processing in the step 23 may be optionally performed by animage processing engine. In a preferred embodiment, the image processingengine may be an artificial intelligence image processing engine, whichutilizes a machine learning algorithm, one or more knownthree-dimensional images, and a plurality of training planar imagesprojected from the known three-dimensional images for optimization. Thesaid machine learning algorithm may be an iterative convolutional neuralnetwork algorithm. As for the plurality of training planar images, anyof which is an X-ray planar image containing posture parameters of theskeleton and/or joint in the three-dimensional images. For example, theX-ray planar image used for training can be an X-ray planar image (suchas a hospital medical record image) exposed by a general X-ray machine,but undergoes image recognition before it is input to the imageprocessing engine for machine learning, wherein the recognition is toestimate the posture parameters corresponding to the skeleton and/orjoint, and the posture parameters represent the posture variation of theskeleton and the joint. For example, as shown in FIGS. 3A and 3B, whenthe skeletal joint is a knee joint, the posture parameters may comprisethe joint bending angle θ between the femur 30 and the tibia 31 (asshown in FIG. 3A) and the three-axis rotational angles x, y, z to theknee joint itself (shown in FIG. 3B). As such, the machine learningalgorithm may be simultaneously trained according to the X-ray planarimage and various posture parameters, so that further optimize theability of the image processing engine to generate the stereoscopicimage data file. In addition, a plurality of X-ray planar images fortraining may also be generated from a known stereoscopic image data fileset at different angles, and the known stereoscopic image data file canbe an image data files that have been completed by any means (such ascomputed tomography).

Accordingly, the image processing in step 23 may comprise the followingsteps: estimating a set of first posture parameters according to thefirst X-ray planar image, estimating a set of second posture parametersaccording to the second X-ray planar image, and generating astereoscopic image data according to the first X-ray planar image, thesecond X-ray planar image, the set of first posture parameters, and theset of second posture parameters. The first and second X-ray planarimages are firstly subjected to image recognition to calculate theposture parameters corresponding to the skeletal joint. For example, asshown in FIG. 3A and FIG. 3B, when the skeletal joint is a knee joint,the posture parameters may comprise the joint bending angle θ betweenthe femur 30 and the tibia 31 (FIG. 3A) and the three-axis rotationalangles x, y, z of the knee joint itself (FIG. 3B). As can be seen fromthe above, the image processing may also generate a stereoscopic imagedata file according to the X-ray planar image and the posture parametersat the same time, and the resulting stereoscopic image data file will becloser to its real appearance.

In addition, since the brightness of each pixel in the X-ray planarimage shown in FIG. 1D represents the tissue density of the point, thereconstructed 3D image data file can be a collection of voxelsrepresenting tissue density. That is, there is also an intensity value(I) corresponding to the three-dimensional coordinates (X, Y, Z) foreach voxel point.

Since the reconstructed image file in present invention is astereoscopic image data file, the image display 11 shown in FIG. 1 maybe a general flat-panel display, and the image processor 10 may generatea plurality of planar images from different angles of the stereoscopicimage. The flat-panel display may then be used to switch and display theabove-mentioned planar images of different angles, so that the viewercan have a feeling of three-dimension (e.g. rotational display fromvarious angles). In addition, the image display 11 may also be a devicelike a virtual reality display or an augmented reality display, whichmay be used to receive the stereoscopic image data file to directlydisplay a 3D image with a stereoscopic sense. In this way, the disclosedapparatus may allow doctors and patients to fully communicate thepatients' conditions. In addition, the method of the present applicationmay further comprise the following steps: utilizing the stereoscopicimage data file to estimate a physiological anatomical feature of theskeleton or joint (such as the size of the bone), the position, or therelative geometric relationship in the three-dimensional space. Theabove information can be used for disease diagnosis, surgical evaluationor assistive device design. For example, the relative geometricrelationship of the skeleton and joint may be an inter-joint bonedistance data or the proportional size of a group of specific bones. Theinter-joint bone distance data may be knee joint distance data, and theproportional size of the specific group of bones may be the proportionsof the femur, tibia, patella and fibula in the knee joint. According tothe above information, the doctor may be able to determine the stage ofknee osteoarthritis, and provide appropriate advice and treatment.

Furthermore, each voxel in the estimated stereoscopic image data file inthe present invention may comprise a set of three-dimensional coordinatedata and a characteristic value, and the characteristic value mayrepresent tissue density. Moreover, after the stereoscopic image file isprocessed by image recognition, a group of skeleton model annotationscan be defined (for example, a certain part of the stereoscopic imagefile is automatically marked as the name of a certain bone). In thisway, as shown in FIG. 4 , the present invention may further comprise athree-dimensional object forming device 19 (such as a 3D printer), whichis electrical connected to the image processor 10 by signals forreceiving the three-dimensional image data file. The three-dimensionalobject forming device 19 may fabricate a three-dimensional object (suchas a certain bone or a repair part in a joint) with a restored tissuedensity according to the bone model annotation in the three-dimensionalimage data file. In addition, the voxel set that can represent thetissue density may also project a two-dimensional image with a densitydistribution state. Both the voxel set or the two-dimensional image withthe density distribution state may be used to estimate the bone strengthor fracture risk at that position.

EXAMPLE

The following example is provided to further illustrate the imageprocessing method as claimed.

1. Interference Image Removal

In step 22, The two X-ray two dimensional images (the first X-raytwo-dimensional image 131 and the second X-ray two-dimensional image132) are obtained by a normal X-ray machine. Both the first interferenceimage and the second interference image are removed from the X-rayimages for a better reconstruction quality.

The first step of interference image removal utilizes a U-net neuralnetwork to segment the bone area of the input image. The U-net is awell-known convolutional neural network architecture, which is trainedin advance by a training model using a bone data collection withlabeling.

The second step of interference image removal takes the pixel valuesaround the bone contour, predict soft tissue values over the region bysolving a Laplace equation and then subtract the soft tissue values fromthe input image, as described by Gong in journal article titled“Decompose X-ray Images for Bone and Soft Tissue” (arXiv:2007.14510v1).In brief, with the input X-ray image f(x, y), obtain the mask M(x, y) byactive contour or user input, then compute the soft tissue interferenceimage S(x, y) by solving the equation below:

ΔS _(M)=0,s.t. S _(∂M) =f _(∂M),  (Eq. 1)

where ∂ denotes the boundary. After calculating S(x, y), compute α valueby the following equation:

$\begin{matrix}{\alpha \equiv \frac{1}{\max\left\{ \frac{{f\left( {x,y} \right)} - {S\left( {x,} \right)}}{1 - {S\left( {x,y} \right)}} \right\}}} & \left( {{Eq}.2} \right)\end{matrix}$

Lastly, compute the soft tissue interference image removed bone imageU(x, y) by the equation below:

$\begin{matrix}{{U\left( {x,y} \right)} = {\alpha\frac{{f\left( {x,y} \right)} - {S\left( {x,y} \right)}}{1 - {S\left( {x,y} \right)}}}} & \left( {{Eq}.3} \right)\end{matrix}$

The U(x, y) described in Eq. 3 is the desired bone image withinterference image removed. FIG. 5 shows the interference image removalprocess by the above method.

The interference image removal described is an important step to producegood quality 3D image, as shown in FIG. 6 . FIG. 6A is the CT referencewhich generates the two input images, and FIG. 6B-6E are the constructed3D images using the two generated input images. The multi-scalestructural similarity (MS-SSIM) values are also provided below eachfigure. FIG. 6B shows the construction result of two 2D X-ray imageswith both interference images retained, FIGS. 6C and 6D are theconstruction result with one of the interference images removed, andFIG. 6E is the result where both interference images are removed. Fromthe above result, it is clear that removing the interference images fromthe input is essential for constructing good quality images.

2. Three-Dimensional Image Generation 2.1 Training of ArtificialIntelligence 2.1.1 Training Data Collection

The image processing procedure in step 23 is performed by an artificialintelligence image processing engine. The artificial intelligence imageprocessing engine is as described by Ying et al. in journal articletitled “X2CT-GAN: Reconstructing CT from Biplanar X-Rays with GenerativeAdversarial Networks” (arXiv:1905.06902v1), except that the AI alsoimplements posture parameters prediction and uses the prediction resultsin 3D image construction.

In detail, the AI is trained by sets of artificial X-ray two-dimensionalimages which are generated from corresponding CT images. The trainingdata are pairs of anterior-posterior X-ray image, lateral X-ray image,posture parameters x, y, z, θ, and ground truth CT 3D volume. Theanterior-posterior X-ray image and lateral X-ray image are generated bydigitally reconstructed radiograph (DRR) from ground truth CT 3D volume.This process is to project all data points in CT 3D volume to a 2Dsurface with respect to a camera point using simple trigonometriccalculations.

To generate training data with rotation parameters (x, y, z), the groundtruth CT 3D volume is rotated with respect to x-axis, y-axis, and z-axisby different amounts of angles (x, y, z). The rotated 3D volume forms apoint cloud in three-dimensional space. Then, a DRR is generated byprojecting from a camera point through all points in the 3D point cloudto a 2D surface. FIG. 7 shows an example of generated training 2Dsurface with labeled posture parameters. The training data of 2Dsimulated X-ray paired with known posture parameters by projecting 3D CTimages with all kinds of different parameters are used to train aregression neural network.

The knee joint bending parameter θ is generated by manipulating thecomponents of the ground truth CT 3D volume. Since CT are commonlyscanned while knee joint are stretched straight, by rotating the femurcomponent or the tibia component with respect to the knee joint centeraxis, the 3D volume of bended knee is simulated. Then DRRs withdifferent rotation parameters using the θ-bended 3D volume aregenerated.

2.1.2 Loss Calculation in Model

FIG. 8 -FIG. 10 show 3 different embodiments to implement the training.As those figure show, two losses are calculated during training. Loss 1is the loss from posture parameters, we can calculate mean absoluteerror or mean squared error as the loss function during training:

loss_(MAE)=¼(|x _(pred) −x _(true) |+|y _(pred) −y _(true) |+|z _(pred)−z _(true)|+θ_(pred)−θ_(true)|)

loss_(MSE)=¼((x _(pred) −x _(true))²+(y _(pred) −y _(true))²+(z _(pred)−z _(true))²+(θ_(pred)−θ_(true))²)

Loss 2 is the loss from generated 3D volume, we can also calculate meanabsolute error or mean squared error between the predicted and groundtruth 3D volume as the loss function:

${{loss}_{MAE} = {\frac{1}{V_{N}}{\sum_{i = 0}^{V_{N}}{❘{V_{{pred}_{i}} - V_{{true}_{i}}}❘}}}},{{loss}_{MSE} = {\frac{1}{V_{N}}{\sum_{i = 0}^{V_{N}}\left( {V_{{pred}_{i}} - V_{{true}_{i}}} \right)^{2}}}}$

In addition, loss 2 also added projection loss which encourages theprojection images on each three dimensions between predicted and groundtruth volumes are alike.

2.1.3 Posture Parameters Application in Model

In embodiment 1 (FIG. 8 ) and 3 (FIG. 10 ), posture parameters areapplied when combining two 3D feature matrices derived from input 1 andinput 2. Conventional method rotates one of the matrix 90 degreesinvariably before summing them together. The method of the presentinvention, however, considers the rotation parameters, rotating bothmatrices with respect to x-axis, y-axis, and z-axis according to theparameters to make them orthogonal. This leads to a more ideal result ifthe input X-ray images are not filmed rightly in shooting angle. Also,the upper and lower part of the matrices can be rotated by knee jointbending parameter θ with respect to the matrix center axis to retrieve3D volume of straight stretched knee.

Besides directly using the posture parameters to rotate both matrices,it's also possible to concatenate or perform matrix multiplication ofthose parameters with specific layer(s) of the convolutional neuralnetwork (CNN) before combining the 3D feature matrices derived frominput 1 and input 2. This leaves the model itself to learn the bestcombining weights from the posture parameters to reconstruct the final3D volume.

In embodiment 2 (FIG. 9 ), the posture parameters are applied on 2Dinput X-ray images. The input images are adjusted into other images asif they were shot orthogonally by doing image affine transformation, oragain concatenate or perform matrix multiplication of those parameterswith specific layer(s) of the convolutional neural network (CNN), makingthe layers learn the orthogonal images.

2.2 Three-Dimensional Image Generation and the Results

During 3D image construction, the CNN model described by Ying et al.(arXiv:1905.06902v1) is implemented, but the matrix is rotated accordingto the posture parameters instead of 90 degrees. The “Connection-C”procedure described in Ying et al. is to rotate one of the 3D matrix 90degrees before summing together. In the present invention, the matrix isrotated according to the predicted posture parameter values rather than90 degrees.

The application of the posture parameter makes the model more robustwhen the two input X-ray images are not orthogonal, as illustrated inFIG. 11 . FIG. 11A is the CT reference used to generate input images,and FIG. 11B-11D are the output 3D images. FIG. 11B is an output 3Dimage constructed from two orthogonal input images. FIG. 11C is anoutput 3D image constructed from two non-orthogonal input images withoutconsideration of the posture parameters, i.e. the two input images areprocessed as if they were orthogonal to each other. The output image inFIG. 11C is skewed. FIG. 11D is an output 3D image constructed from twonon-orthogonal input images with consideration of the predicted postureparameters, and the output image is normal (not skewed) compared to FIG.11C. The MS-SSIM values are also provided below each figure. The resultindicates that the inclusion of predicted posture parameters improvesthe quality of generated 3D image when the angle of the two input imagesdeviates from 90 degrees, as shown from the MS-SSIM values. The resultof MS-SSIM performance versus rotation angle is shown in FIG. 12 ,indicating that the model's performance degraded significantly if anglecorrection was not applied.

The 3D image generated by the claimed method is with high fidelity. Thecomparison between generated 3D image and real CT 3D image is evaluatedby multi-scale structural similarity (MS-SSIM) index. The MS-SSIM amonga test set with 22 test data is 0.746±0.0955. FIG. 13 also provides anexample of comparison between generated 3D image and CT 3D image, withMS-SSIM value 0.81641.

The disclosed 3D image generation method not only provide a method togenerate the contour of a 3D image from 2D images, but also provides theintensity of each voxel in the constructed 3D image. As described inprevious paragraphs, the brightness of each pixel in the planar imagesrepresents the tissue density at that point, so the stereoscopic imagedata reconstructed from the two X-ray planar images by the method is acollection of voxels that representing tissue density at each point.That is, there is also an intensity value (I) corresponding to thethree-dimensional coordinates (X, Y, Z) of each voxel point, as shown inFIG. 14 . The two images on the left are the input images, and the twoimages on the right are the generated three-dimensional image. Thewhiteness (opacity) in the images represents the intensity values, whichare also recorded in the output data.

The foregoing description of embodiments is provided to enable anyperson skilled in the art to make and use the subject matter. Variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the novel principles and subject matterdisclosed herein may be applied to other embodiments without the use ofthe innovative faculty. The claimed subject matter set forth in theclaims is not intended to be limited to the embodiments shown herein butis to be accorded the widest scope consistent with the principles andnovel features disclosed herein. It is contemplated that additionalembodiments are within the spirit and true scope of the disclosedsubject matter. Thus, it is intended that the present invention coversmodifications and variations that come within the scope of the appendedclaims and their equivalents.

What is claimed is:
 1. A method for generating three-dimensional imageof human bones, comprising the steps of: providing a first X-ray planarimage and a second X-ray planar image captured from a first angle and asecond angle of the human bone; predicting, by an image processingengine, one or more sets of predicted posture parameters for the firstX-ray planar image and the second X-ray planar image; and generating, bythe image processing engine, the data of a stereoscopic image accordingto the first X-ray planar image, the second X-ray planar image, and thepredicted posture parameters.
 2. The method of claim 1, beforepredicting posture parameters further comprising: removing a firstinterference image from the first X-ray planar image; and removing asecond interference image from the second X-ray planar image.
 3. Themethod of claim 2, wherein each of the first interference image and thesecond interference image is the background interfering image resultingfrom non-skeletal objects.
 4. The method of claim 1, wherein the imageprocessing engine utilizes a machine learning algorithm, one or moreknown three-dimensional (3D) validating images, and a plurality oftwo-dimensional (2D) training images which are projections of the known3D validating images to optimize the ability to generate the data of astereoscopic image.
 5. The method of claim 4, wherein the plurality of2D training images is digitally reconstructed from projecting the known3D validating images at different angles.
 6. The method of claim 5,wherein the known 3D validating images are CT images.
 7. The method ofclaim 4, wherein each of the plurality of 2D training images comprises aset of training posture parameters.
 8. The method of claim 7, whereinthe training posture parameters comprise the rotation angles for each ofthe plurality of 2D training images around x, y and z axis.
 9. Themethod of claim 8, further comprising an angle θ representing thebending angle of a joint.
 10. The method of claim 1, wherein thepredicted posture parameters comprise a set of first predicted postureparameters for the first X-ray planar image and a set of secondpredicted posture parameters for the second X-ray planar image.
 11. Themethod of claim 10, wherein the predicted posture parameters comprisethe rotation angles for each of the first X-ray planar image and thesecond X-ray planar image around x, y and z axis.
 12. The method ofclaim 11, further comprising an angle θ representing the bending angleof a joint in the first X-ray planar image and the second X-ray planarimage.
 13. The method of claim 12, wherein both the set of first postureparameters and the set of second posture parameters comprise the jointbending angle between femur and tibia, and the three-axis rotation angleof the knee joint itself.
 14. The method of claim 4, wherein the machinelearning algorithm is a convolution neural network (CNN).
 15. The methodof claim 1, wherein the data of the stereoscopic image are a set ofvoxels representing bone density of the corresponding positions.
 16. Themethod of claim 15, wherein the data of the stereoscopic image comprisesa set of bone model annotation.
 17. A machine learning method fortraining a machine with image data of human bones, comprising: providingone or more three-dimensional (3D) validating image associated with thehuman bones; providing a plurality of two-dimensional (2D) trainingimages, each of which is a projected image generated from one of the 3Dvalidating images in a specific angle defined as an angle parameterassociated with the 2D training image; and training the machine with the3D validating images, the plurality of 2D training images, and the angleparameters associated with the plurality of 2D training images, whereinthe machine after training is able to generate a 3D target image fromtwo 2D input images representing different projections of the 3D targetimage.
 18. The method of claim 17, wherein the training step is trainedin a convolution neural network (CNN).